Binary classification by minimizing the mean squared slack

The paper presents a new binary classification method based on the minimization of the slack variables energy called the Mean Squared Slack (MSS). We deliver preliminary mathematical results which support the motivation behind our approach. We show that (a) in the linearly separable case the minimum MSS is attained at a separating vector, while (b) the minimizer in the linearly non-separable case is bounded but not zero. The method is conceptually simple: it solves a linear system at each iteration and it converges, typically, within a few iterations. Its complexity is obviously related to the size of the system which, in the linear case, is equal to the input pattern dimension. The method is extended to the non-linear case using kernels. Simulations demonstrate that the method is competitive with respect to computation time, accuracy, and generalization performance compared to state of the art SVM methods. © 2012 IEEE

Towards minimizing the energy of slack variables for binary classification

This paper presents a binary classification algorithm that is based on the minimization of the energy of slack variables, called the Mean Squared Slack (MSS). A novel kernel extension is proposed which includes the withholding of just a subset of input patterns that are misclassified during training. The later leads to a time and memory efficient system that converges in a few iterations. Two datasets are exploited for performance evaluation, namely the adult and the vertebral column dataset. Experimental results demonstrate the effectiveness of the proposed algorithm with respect to computation time and scalability. Accuracy is also high. In specific, it equals 84.951% for the adult dataset and 91.935%, for the vertebral column dataset, outperforming state-of-the-art methods. © 2012 EURASIP